Queryless search based on context

ABSTRACT

Computer-readable media, computer systems, and computing devices for initiating a queryless search is provided. In this regard, a search can be initiated without a user inputting a query. In one embodiment, the method includes recognizing a search indication provided by a user to indicate a desire to view one or more search results related to content being presented. Such a search indication can be performed without accessing a web page associated with a search engine. In accordance with the search indication, contextual information related to the content being presented can be automatically captured. Thereafter, the captured contextual information is provided to initiate a search for information corresponding with the contextual information. Search results corresponding with the contextual information are received and presented.

BACKGROUND

In order to find desired content, a computer user often makes use of asearch service. Exemplary Internet search engines are well known in theart, for example, a commonly known commercial engine is the BING searchengine provided by Microsoft Corporation of Redmond, Wash. Generally, toutilize a search service, a user initially opens a web browser andnavigates to a home page associated with the particular search service.Thereafter, the user can enter a search query to obtain desired searchresults, for example, via a search box provided by the search service.Upon receiving search results related to the user-input search query,the user can sift through results to find desired information. Such asearch process can be time consuming and inefficient, particularly if auser cannot particularly articulate a search query to obtain desiredinformation.

SUMMARY

This summary introduces simplified concepts of facilitating querylesssearching, which is further described below in the Detailed Description.This summary is not intended to identify essential features of theclaimed subject matter, nor is it intended for use in limiting the scopeof the claimed subject matter.

This application describes example embodiments of performing a querylesssearch based on a search indication provided by a user. In one exampleembodiment, a user may be viewing content via a user computing device.Such content may be any content that can be displayed via a displayscreen of a user device. For example, the user device may displaytextual content on a display to a user or may play a video and/or anaudio to the user. The user device may subsequently receive a searchindication from the user that indicates an intent to initiate a searchusing contextual information. Upon receiving such a search indication,contextual information can be automatically captured and utilized toinitiate a search. In this regard, the user can view search resultsrelated to the contextual information captured by the user devicewithout inputting a user search query.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention are described in detail below withreference to the attached drawing figures, wherein:

FIG. 1 is a block diagram of an exemplary computing device suitable forimplementing embodiments of the invention;

FIG. 2 is a schematic diagram of an exemplary environment including auser device usable to implement queryless searches, in accordance withan embodiment of the present invention;

FIG. 3 is a schematic diagram that illustrates a first example in whicha queryless search is performed in association with a social networkingapplication, in accordance with an embodiment of the present invention;

FIG. 4 is a schematic diagram that illustrates a second example in whicha queryless search is performed in association with a social networkingapplication, in accordance with an embodiment of the present invention;

FIG. 5 is a schematic diagram that illustrates a third example in whicha queryless search is performed in association with a social networkingapplication, in accordance with an embodiment of the present invention;

FIG. 6 is a schematic diagram that illustrates an example in which aqueryless search is performed in association with a video hostingapplication, in accordance with an embodiment of the present invention;

FIG. 7 is a schematic diagram that illustrates an example of a displayedtelevision program, in accordance with an embodiment of the presentinvention;

FIG. 8 is a schematic diagram that illustrates an example of aninitiation of a queryless search according to an embodiment of thepresent invention;

FIG. 9 is a schematic diagram that illustrates an example of searchresults presented based on performance of a queryless search, inaccordance with an embodiment of the present invention;

FIG. 10 illustrates a first example for facilitating a queryless search,in accordance with embodiments of the present invention;

FIG. 11 illustrates a second example for facilitating a querylesssearch, in accordance with embodiments of the present invention; and

FIG. 12 illustrates a third example for facilitating a queryless search,in accordance with embodiments of the present invention.

DETAILED DESCRIPTION

Typically, when a user wants to search for certain information relatedto content a user is viewing (e.g., watching a television program,watching a video or video clip, browsing the web, reading an email,working on a word processing document, etc.), the user opens a webbrowser on a user computing device and connects with a search service.The user may query a search engine by entering a search query. Such asearch query may be entered via a search box which is integrated with aweb browser and/or webpage or run with the web browser as a toolbar, orthrough a separate application that uses the web browser as a platformto complete a search. In either case, the user manually enters thesearch query, either by typing or by cutting and pasting the searchquery from the task he/she was previously working on. Other existingtechnologies may allow a user to input a search query by activating avoice recognition function of the user's device, if available, to inputvoice commands and/or voice inputs, which are converted into text by asearch application of the user's device or the search service.

In response to receiving the search query, the search service may returnand present search results to the user on a results page. If the userwants to modify the search, the user may enter a modified search queryinto the text box and submit the modified search query to the searchengine, which then returns new search results to the user. Once the userfinds the desired information, he or she then exits the search serviceand returns to the task he/she was originally performing. Should theuser need to refer to the search results or perform another search, theuser must switch back and forth between his/her original task and thesearch service, thus disrupting his or her workflow.

The situation becomes even worse if a user is utilizing a user devicethat does not readily have functionality to facilitate a search andpresent search results, for example, via a web browser while the user isutilizing the user device. For example, assume a user is watchingtelevision and is interested in an actor or event presented via thetelevision. In such a case, the user would generally access another userdevice, such as a laptop or mobile device, access a web page hosted by asearch service (e.g., via a web browser), formulate a user query, enterthe query, and sift through the search results. Further, formulating thequery may be quite difficult if the user is unfamiliar with the name ofthe actor, event, or other details that might assist in providing thedesired information.

In another example, a user may be viewing content (e.g., via a web page)that includes an image. The user may want to conduct a search for theimage. To conduct the search, the user may need to know how to describethe image and provide a textual description of the image to a searchengine, similar to the search process described above. Alternatively,the user may first save the image to local memory of the device and thenupload the image to a search engine that allows content based imageretrieval. In both cases however, the user is required to go through anumber of steps in order to perform the image search. Furthermore, somemobile devices such as mobile phones having limited memory and/orfunctional capabilities may not allow storage of an image from theInternet into the mobile device or may require cumbersome procedures todo so. These are just some examples of difficulties that exist withsearching using mobile phones and other client devices.

Embodiments of the present invention are directed to facilitatingqueryless searches, which allow a search to be performed in response toa user indication without additional input from a user. In this regard,aspects of the present invention enable a user to be presented withinformation related to content being viewed by the user without the userbeing required to formulate and/or input a query. That is, to view datarelated to content being viewed by a user, the user may simply providean indication of such a desire without inputting or submitting a query,such as a text query. In embodiments, a queryless search is performedusing only contextual information derived by a user computing device.

Generally, a user device displays or causes display of content to a uservia a display screen. Such content may be any content that can bedisplayed via a display screen of a user device. For example, the userdevice may display textual content on a display to a user or may play avideo and/or an audio to the user. The user device may subsequentlyreceive a search indication from the user that indicates an intent toinitiate a search using contextual information. Upon receiving such asearch indication, contextual information can be captured and utilizedto initiate a search. In this regard, the user can view search resultsrelated to the contextual information captured by the user devicewithout inputting a user search query.

Embodiments of the invention described herein include acomputer-implemented method of facilitating a queryless search. Themethod includes recognizing a search indication provided by a user toindicate a desire to view search results related to content beingpresented. Such a search indication is performed without accessing a webpage associated with a search engine. In accordance with the searchindication, contextual information related to the content beingpresented is automatically captured. The contextual information isautomatically provided to initiate a search for informationcorresponding with the contextual information. Thereafter, searchresults corresponding with the contextual information are received.

In a second illustrative embodiment, one or more computer-readable mediastoring instructions that, when executed by a processor, configure theprocessor to perform acts. The acts include detecting a searchindication comprising an action performed by a user in association witha controller. In response to detecting the search indication, contextualinformation related to media content being displayed at or neardetection of the search indication is automatically obtained. A searchto be performed is initiated using the contextual information byautomatically communicating the contextual information at least to aremote search service that formulates a query based on the contextualinformation. Search results are received related to the obtainedcontextual information.

In a third illustrative embodiment, a user device includes a processor;memory communicatively coupled to the processor; and a display screenconfigured to display content. The user device also includes a searchdetector component to detect a search indication that provides anindication to perform a search using contextual information, wherein thesearch indication does not specify any text for use in performing thesearch. The user device further includes a context capturer component toautomatically capture contextual information in accordance withdetecting the search indication, wherein at least a portion of thecontextual information corresponds with a time at or near detection ofthe search indication. The user device additionally includes a searchpresenting component to cause presentation of one or more search resultsthat correspond with the contextual information, wherein the one or moresearch results are presented in association with the displayed content.

Multiple and varied implementations and embodiments are described below.The following section describes an example environment that is suitablefor implementing gesture-based search. The following sections describeillustrative aspects of the gesture-based search techniques.

Various aspects of embodiments of the invention may be described in thegeneral context of computer program products that include computer codeor machine-useable instructions, including computer-executableinstructions such as program modules, being executed by a computer orother machine, such as a personal data assistant or other handhelddevice. Generally, program modules including routines, programs,objects, components, data structures, etc., refer to code that performparticular tasks or implement particular abstract data types.Embodiments of the invention may be practiced in a variety of systemconfigurations, including dedicated servers, general-purpose computers,laptops, more specialty computing devices, and the like. The inventionmay also be practiced in distributed computing environments where tasksare performed by remote-processing devices that are linked through acommunications network.

Computer-readable media include both volatile and nonvolatile media,removable and nonremovable media, and contemplate media readable by adatabase, a processor, and various other networked computing devices. Byway of example, and not limitation, computer-readable media includemedia implemented in any method or technology for storing information.Examples of stored information include computer-executable instructions,data structures, program modules, and other data representations. Mediaexamples include, but are not limited to RAM, ROM, EEPROM, flash memoryand other memory technology, CD-ROM, digital versatile discs (DVD),holographic media and other optical disc storage, magnetic cassettes,magnetic tape, magnetic disk storage, and other magnetic storagedevices. These technologies can store data momentarily, temporarily, orpermanently.

An exemplary operating environment in which various aspects of thepresent invention may be implemented is described below in order toprovide a general context for various aspects of the present invention.Referring initially to FIG. 1, an exemplary operating environment forimplementing embodiments of the present invention is shown anddesignated generally as computing device 100. The computing device 100is but one example of a suitable computing environment and is notintended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should the computing device 100be interpreted as having any dependency or requirement relating to anyone or combination of components illustrated.

The computing device 100 includes a bus 110 that directly or indirectlycouples the following devices: a memory 112, one or more processors 114,one or more presentation components 116, input/output (I/O) ports 118,input/output components 120, and an illustrative power supply 122. Thebus 110 represents what may be one or more busses (such as an addressbus, data bus, or combination thereof). Although the various blocks ofFIG. 1 are shown with lines for the sake of clarity, in reality,delineating various components is not so clear, and metaphorically, thelines would more accurately be gray and fuzzy. For example, one mayconsider a presentation component such as a display device to be an I/Ocomponent. Also, processors have memory. The inventors recognize thatsuch is the nature of the art, and reiterate that the diagram of FIG. 1is merely illustrative of an exemplary computing device that can be usedin connection with one or more embodiments of the present invention.Distinction is not made between such categories as “workstation,”“server,” “laptop,” “hand-held device,” “tablet,” etc., as all arecontemplated within the scope of FIG. 1 and reference to “computingdevice.”

The memory 112 includes computer-executable instructions (not shown)stored in volatile and/or nonvolatile memory. The memory may beremovable, nonremovable, or a combination thereof. Exemplary hardwaredevices include solid-state memory, hard drives, optical-disc drives,etc. The memory 112 is an example of computer readable media.Computer-readable media includes at least two types of computer readablemedia, namely computer storage media and communications media.

Computer storage media includes volatile and non-volatile, removable andnon-removable media implemented in any method or technology for storageof information such as computer readable instructions, data structures,program modules, or other data. Computer storage media include, but arenot limited to, phase change memory (PRAM), static random-access memory(SRAM), dynamic random-access memory (DRAM), other types ofrandom-access memory (RAM), read-only memory (ROM), electricallyerasable programmable read-only memory (EEPROM), flash memory or othermemory technology, compact disk read-only memory (CD-ROM), digitalversatile disks (DVD) or other optical storage, magnetic cassettes,magnetic tape, magnetic disk storage or other magnetic storage devices,or any other non-transmission medium that can be used to storeinformation for access by a computing device.

In contrast, communication media may embody computer readableinstructions, data structures, program modules, or other data in amodulated data signal, such as a carrier wave, or other transmissionmechanism. As defined herein, computer storage media does not includecommunication media.

The computing device 100 includes one or more processors 114 coupledwith a system bus 110 that read data from various entities such as thememory 112 or I/O components 120. In an embodiment, the one or moreprocessors 114 execute the computer-executable instructions to performvarious tasks and methods defined by the computer-executableinstructions. The presentation component(s) 116 are coupled to thesystem bus 110 and present data indications to a user or other device.Exemplary presentation components 116 include a display device, speaker,printing component, and the like.

The I/O ports 118 allow computing device 100 to be logically coupled toother devices including the I/O components 120, some of which may bebuilt in. Illustrative components include a microphone, joystick, gamepad, satellite dish, scanner, printer, wireless device, keyboard, pen,voice input device, touch-input device, touch-screen device, interactivedisplay device, or a mouse. The I/O components 120 can also includecommunication connections that can facilitate communicatively connectingthe computing device 100 to remote devices such as, for example, othercomputing devices, servers, routers, and the like.

FIG. 2 is a schematic diagram of an example environment 200 usable tofacilitate viewing search results related to items within content. Theenvironment 200 includes a user device 202 and a search service 204 incommunication with one another via a network 206. The user device 202and the search service 204 can communicate through a network 206 or acombination of networks (not shown), which may include any number ofnetworks such as, for example, a local area network (LAN), a wide areanetwork (WAN), the Internet, a cellular network, a peer-to-peer (P2P)network, a mobile network, or a combination of networks.

The network environment 200 shown in FIG. 2 is an example of onesuitable network environment and is not intended to suggest anylimitation as to the scope of use or functionality of embodiments of theinventions disclosed throughout this document. Neither should theexemplary network environment 200 be interpreted as having anydependency or requirement related to any single component or combinationof components illustrated therein. For example, the user device 202 andthe search service 204 may be integrated in a single component (e.g.,the user device 202) or may directly communicate with one another.

The user device 202 can be any kind of computing device or devicescapable of initiating a queryless search. For example, the user device202 can be a computing device such as computing device 100. The userdevice 202 may be implemented as any one of a variety of conventionalcomputing devices such as, for example, a desktop computer, a notebookor laptop computer, a netbook, a tablet or slate computer, a surfacecomputing device, an electronic book reader device, a workstation, amobile device (e.g., Smartphone, personal digital assistant, in-carnavigation device, etc.), a game console, a set top box, or acombination thereof. Depending on the type of user device, user devicemay include, for example, a touch screen or other display, a keyboard, amouse, a touch pad, a roller ball, a scroll wheel, an image capturedevice, an audio input device, an audio output device, and/or any otherinput or output devices.

The user device 202 may include any number of applications that arecapable of, among other things, displaying content to a user on adisplay of the user device 202 or on a display associated with orconnected to the user device 202. By way of example and not limitation,applications may include a desktop environment (e.g., icons, windows,toolbars, folders, wallpapers, desktop widgets, etc.), a web browser, anemail client, a word processing application, a spreadsheet application,a voice recording application, a calendaring application, a newsapplication, a text messaging client, a media player application, aphoto album application, an address book application, a viewfinderapplication, a social networking application or a game.

In embodiments, the user device 202 includes a search detector 210, acontext capturer 212, and a search result presenter 214. In someembodiments, one or more of the illustrated components/modules may beimplemented as stand-alone applications. In other embodiments, one ormore of the illustrated components/modules may be implemented via anoperating system or integrated with an application running on a device.It will be understood by those of ordinary skill in the art that thecomponents/modules illustrated in FIG. 2 are exemplary in nature and innumber and should not be construed as limiting. Any number ofcomponents/modules may be employed to achieve the desired functionalitywithin the scope of embodiments hereof. Further, components/modules maybe located on any number of computing devices.

The search detector 210 of the user device 202 is configured to receive,recognize, or detect a search indication. A search indication refers toany indication to initiate and/or perform a search using contextualinformation. In this regard, a user utilizing the user device 202 canprovide a search indication to initiate a queryless search in any of avariety of manners. As such, a search indication is performed withoutaccessing a web page associated with a search engine or a searchapplication configured to receive a user query.

The search indication may include, but is not limited to, touch input toa touch screen, touch pad, button associated with the user device; amotion of the user device; a body motion detected by an image capturedevice of the user device; a motion of a component associated with theuser device (e.g., a remote control); a motion of a cursor controlled bya mouse or other user input device; and/or a voice command.

With regard to a touch input to a touch screen, touch pad, button, etc.associated with a user device, the search detector 210 can be configuredto detect such user interaction. That is, touch input from one or morefingers of the user or a pointing device such as a stylus or digital pencan provide a search indication. For example, a particular button on aremote control (e.g., a mechanical button or a touch button), ifselected, can be configured to initiate a search based on contextualinformation. In another example, a touch input of an icon or link caninitiate a search based on contextual information (e.g., an icon on atoolbar or desktop, etc.).

Additionally or alternatively, the search indication may include amotion of the user device itself. The motion of the user device may bedetected by, for example, an accelerometer, an attitude sensor, acompass, etc. The motion of the user device may indicate an intent ofthe user to initiate a search related to the content being presented viathe user device. Any predetermined motion could be used to signify asearch indication.

In additional or alternative embodiments, the search detector 210 mightbe configured to detect a body motion, for example, using an imagecapture device of the user device. In this regard, the search indicationmay include a motion of a body or a part of the body of the user such asa finger, a hand, head, eye, and/or an arm. Such body motion may bedetected using a camera, other image capture device, or any motiondetection component of the user device 202. For example, the user devicemay be a gaming device with a camera or other motion detection interfacesuch as an Xbox® gaming console configured with a Kinect™ motiondetection system, both available from Microsoft Corporation of RedmondWash. The user device may receive and interpret images or signals theuser device received to determine what motion the user is performing.The motion of the user may provide an indication of a desire to performa search based on contextual information in association with the contentbeing viewed by the user. Any predetermined motion could be used tosignify a search indication.

Similarly, the search detector 210 might be configured to detect amotion of a component associated with the user device (e.g., a remotecontrol), for example, using an image capture device of the user device.In this regard, the search indication may include a motion of acomponent associated with the user device such as a remote control, agame controller, pointer, joystick, etc. Such a motion may be detectedusing a camera, other image capture device, or any motion detectioncomponent of the component associated with the user device 202 (e.g.,accelerometer, gyroscope, etc). For example, the user device may be agaming device with a camera or other motion detection interface. Theuser device may receive and interpret images or signals the user devicereceived to determine what motion the user is performing using acomponent. The motion of the component may provide an indication of adesire to perform a search based on contextual information inassociation with the content being viewed by the user. Any predeterminedmotion could be used to signify a search indication.

Alternatively or additionally, the search detector 210 may detect motionof a cursor controlled by a mouse, track ball, or other input devicecoupled to the user device 202. Accordingly, the search detector 210 maydetect the motion of the cursor controlled by the input device. Further,use of a input device to select an icon, link, etc. can also be used toprovide a search indication (e.g., a selection of an icon on a toolbaror desktop, hovering over an icon, etc.).

In some embodiments, a search indication may indicate or identify asearch scope. For instance, a search indication may indicate aparticular item or content being displayed that is of interest to theuser (e.g., textual information, image information, visual informationof audio information, video information, etc.) Such a search scope canmore narrowly focus the scope of the search initiated by the userwithout the user providing a query. For example, a search indicationmight substantially bound an item being displayed; touch, overlay, oroverlap an item being displayed; or otherwise provide an indication of aparticular item or content of interest to the user. In some examples, amotion may define a search scope. For instance, the region of contentdefined by the motion may be content that is substantially bounded bythe motion (e.g., a generally circular motion), content that is touchedor overlapped by the search indication, or the like. In embodiments, asearch scope provided is non-text. In this regard, the user does notindicate any text to utilize for a search (e.g., by way of user input,such as a keyboard, a touch input, etc. and/or by way of a voice input).Such a non-text search scope may indicate an image or a portion of avideo for use in specifying a search.

In some embodiments, the user device 202 may display a representation ofthe search scope defined by the search indication on the displayedcontent, to provide visual feedback to the user of the search scope. Therepresentation of the search scope may comprise, for example, a lineenclosing an item(s) of interest, a highlighting view of the item(s) ofinterest, and/or a magnifying lens view of the item(s) of interest.

The context capturer 212 is configured to initiate capturing orobtaining of contextual information with which a search can beperformed. Contextual information refers to information that describesor indicates context associated with the presented content. Inembodiments, contextual information is captured at the time orsubstantially near the time the search indication is detected. Suchcontextual information can pertain to the displayed content being viewedby the user, the user of the user device, and/or the user device itself.

Contextual information associated with the displayed content beingpresented by the user device can refer to content being displayed (e.g.,at or near a time of detecting a search indication), content or an itemidentified as a particular search scope, content near a specified searchscope, a title of a displayed content, a uniform resource locator wherethe displayed content is located, an image within the content beingdisplayed, metadata associated with displayed content (e.g., metadatadescribing text, metadata describing audio media, metadata describingvideo media, etc.), an audio portion being presented, a video portionbeing presented, a channel being presented, a time, location, or framebeing presented at or near of detecting the search indication, and/orthe like.

Contextual information associated with displayed content can be obtainedin any number of ways. In some embodiments, the context capturer 212 canperform object, image, facial, audio, or video recognition or accessanother component that can perform such functionality (e.g., anothercomponent within the user device or a remote computing device). Asanother example, the context capturer 212 can access metadata associatedwith the displayed content and capture such metadata or can accessanother component that has metadata associated with the displayedcontent. In yet other embodiments, the context capturer 212 canadditionally or alternatively capture content displayed (e.g., an image,a video clip, text, etc.).

Contextual information associated with the user device can refer to, forexample, information related to the user device, applications running onthe user device, or location of the user device. In this regard, thecontext capturer 212 can capture contextual information includinginformation related to an application used to display the displayedcontent, information associated with other applications being run by theuser device (e.g., active applications), location data of the userdevice (e.g., using GPS data), or the like.

Contextual information associated with the user can refer to any userdata, such as the user profile (e.g., user demographics, userpreferences, etc.), previous user searches, user contacts (e.g., socialnetwork contacts or friends, relationships of contacts), location of theuser (e.g., where is the user positioned in a room), an address book, acontact list, a calendar, call logs, social data gathered from socialnetworks, contacts, telephone logs, text message logs, affiliations andmemberships of the user, files, images, photos, emails, or any othercontent or data stored on or shared via the user device. Contextualinformation can also refer to third-party data. In this regard,contextual information might include, for instance, activities, actions,searches, etc. being performed by users similar to the current user(e.g., similar by way of a network contact or any other means fordetecting similarity, such as past actions, demographics, specified userinterests, etc.). For example, contextual information could include whatuser similar to the current user have done in a similar setting orcontext. Such data could be obtained locally or be provided to thesearch service by a third-party user device.

Further, contextual information associated with the user can also referto contextual information that can be captured on any other computingdevice being used by the user at or near the time of the searchindication. In this regard, assume a user is requesting a search beperformed in association with a television program being presented onhis or her television (e.g., via a set top box). Further assume that theuser is performing Internet searches on his or her mobile device whilewatching the television program. In such a case, the context capturer212 may be configured to communicate with other user computing devicesto capture contextual information associated with other actions beingperformed by the user at or near the time the user provided a searchindication to obtain search results related to content being displayedon a particular device.

By way of example only, assume that a user is watching an awards programpresented via live television and a particular actor is presented withan award. Further assume that the user is unaware of that particularactor, but would like to know more information about the actor, such asa list of shows or movies in which the actor has appeared. As a result,the user provides a search indication which is detected by the searchdetector 210. In such a case, the context capture 212 can capture anytext being displayed in association with the live television program(e.g., actor's name), metadata associated with programming information,an image of the actor, audio associated with the live televisionprogram, a time of the search indication, a time of the live televisionprogram, a channel on which the live television program is presented, auser identifier, a location of the user within the room, and/or thelike. Further, information concerning actions being performed by theuser on other user devices can also be captured and utilized to performa search. Assume that search results for the actor, the movie, and theawards ceremony are identified as relevant. In some cases, theidentified search results are presented to the user. Additionally oralternatively, the three different types of search results can bepresented to the user such that the user can select a type of searchresults most interesting to the user resulting in display of searchresults related to the selected type. The identified search resultsand/or type of search results can be positioned on the display screen inaccordance with relevance or other ranking mechanism.

As previously described, in some cases, a search indication may indicatea search scope, that is, specify an item or content of particularinterest to the user. In such embodiments, the context capturer 212 canbe configured to determine if an object or content is associated with asearch indication. In this regard, the context capturer 212 determinesor identifies whether content has been specified in association with asearch indication. If so, the context capturer 212 can determine that asearch associated with the selected content or object should beinitiated such that relevant search results can be displayed to theuser.

Upon capturing contextual information, the context capturer 212 canprovide the contextual information to the search service 204. Forexample, as illustrated, the context capturer 212 can provide contextualinformation to the search service 204 using the network 206. In otherembodiments, the search service 204, or a portion thereof, can beintegrated with the user device or other device connected thereto suchthat a local search can be performed. In some embodiments, the localsearch may include a search within the application having the content ofinterest. In other embodiments, the local search may include a searchwithin local memory of the entire user device or a designated memory orfolder of the user device.

In embodiments, the search service 204 includes a query formulator 220and a search engine 222. In some embodiments, one or more of theillustrated components/modules may be implemented as stand-aloneapplications. In other embodiments, one or more of the illustratedcomponents/modules may be implemented via an operating system orintegrated with an application running on a device. It will beunderstood by those of ordinary skill in the art that thecomponents/modules illustrated in FIG. 2 are exemplary in nature and innumber and should not be construed as limiting. Any number ofcomponents/modules may be employed to achieve the desired functionalitywithin the scope of embodiments hereof. Further, components/modules maybe located on any number of computing devices. For example, the searchengine 222 may function utilize a set of computing devices.

The query formulator 220 of the search service 204 is configured toformulate queries utilizing the contextual information obtained, forinstance, by the context capturer 212 of the user device. In someembodiments, the query formulator 220 may perform (or access anothercomponent that can perform) content recognition. For instance, in somecases, the query formulator 214 may receive an image and may performimage recognition in order to generate or formulate an appropriatequery. Other functions described as being performed by the contextcapturer 212 may additionally or alternatively be performed by the queryformulator 220. In formulating queries, the query formulator 220 mayutilize all or a portion of the received or determined contextualinformation.

The search engine 222 is configured to perform a search for searchresults related to the contextual information, or query formulatedtherefrom. The search engine 222 might employ any number of computingdevices, services, or engines to identify search results related to thecontextual information. Search results can be any information associatedwith the contextual information or formulated query. In someembodiments, search results can be similar to traditional search resultsprovided by Internet search engines, such as a set of links deemedrelevant to the query. In other embodiments, search results can beinformation that is provided that is deemed relevant. By way of example,and without limitation, search results may include web results,multimedia results, music results, tweets or social posts from theuser's social network, representations of media associated with thecurrently viewed media, information associated with displayed contentsuch as an image or video (e.g., identification of an actor/actress orother movie item), an option to view a movie, an indication of a localcopy of a media (e.g., a movie), advertisements, similar or relatedmedia content, combinations thereof, etc. In some cases, search resultscan be returned via non-visual means (i.e., read aloud by the devicewith or without any screen representation). In some embodiments, asearch result may be an action indication that can be selected to resultin performance of an action. For example, an action indication may be asave action that, if selected, saves displayed content or contextualinformation; a purchase action that, if selected, enables a user topurchase an item associated with the displayed content; a view actionthat, if selected, results in displaying additional informationregarding the displayed content, or a portion thereof; a communicationaction that, if selected, results in information being communicated(e.g., email, social post, text message, etc.), or the like.

As can be appreciated, the search engine 222 might reference a datastore or index (e.g., a local and/or remote data store or index) toidentify, reference, lookup, or determine search results. As such, thesearch engine 222 can utilize the received contextual information, or aquery derived therefrom, to identify related search results deemedrelevant to the user based on the search indication. In someembodiments, the search engine 222 may take the form of an Internetsearch engine. For example, a commonly known commercial engine is theBING search engine provided by Microsoft Corporation of Redmond, Wash.Such a search engine 222 may perform the search or facilitate the searchby accessing other components or devices to obtain relevant searchresults.

Upon obtaining search results, the search engine 222 can provide thesearch results, or a portion thereof, to the search result presenter 214of the user device 202. The search result presenter 214 can present orcause presentation of the search results, or a portion thereof, via adisplay screen of the user device. As can be appreciated, the searchresults can be displayed at any position or location within the displayscreen. In some embodiments, the search results might be displayed at ornear the top of the display view, at the center of the display view,outside of an active window, in association with a display edge, at anoptimal location based on the displayed content, overlaid over displayedcontent, integrated with displayed content, etc. Accordingly, in somecases, the search result presenter may determine a location at which todisplay the search results.

FIGS. 3-9 illustrate various use scenarios possible using querylesssearch implementations. Various implementations associated with a socialnetwork application are provided in FIGS. 3-5. Initially, FIG. 3illustrates an example in which the queryless search is performed usinga user device in association with a social networking application (e.g.,a web-enabled application or non-web enabled application). Assume a useris using a social networking application running on a user device, suchas a laptop, tablet, or other mobile device. The user device may presentsocial networking content 300 on a display of the user device. Thepresented content may include text, an image, audio content, videocontent, and/or the like. The queryless search techniques describedherein are applicable to any application that displays or otherwisepresents content to a user.

Now assume that a user is particularly interested in a friend's post 302regarding a newly released song and, as such, performs a searchindication in association with the post 302. Such a search indicationmay be, for instance, hold or position a finger over the post, hovercursor over the post, select the post using a cursor, etc. Uponrecognizing such a search indication associated with the post 302,capturing contextual information is automatically initiated. Contextualinformation may include, for instance, a social network identifier, aname of a friend or contact that posted the song, a name of the song(e.g., identified via the content displayed or metadata), etc. Inaccordance with a search using the contextual information, or a queryautomatically generated in association therewith, various search results304 can be received and presented at the user device. Search results 304may include, for instance, the name of the song, the album name, theartist name, other information about the song such as lyrics, othersimilar songs, links to videos having the song, a link to download thesong, for instance, to the user's music collection (e.g., using a useridentifier that captured as context), or the like.

FIG. 4 illustrates another example in which the queryless search isperformed using a user device in association with a social networkingapplication (e.g., a web-enabled application or non-web enabledapplication). Assume a user is using a social networking applicationrunning on a user device, such as a laptop, tablet, or other mobiledevice. The user device may present social networking content 400 on adisplay of the user device. The presented content may include text, animage, audio content, video content, and/or the like. The querylesssearch techniques described herein are applicable to any applicationthat displays or otherwise presents content to a user.

Now assume that a user is particularly interested in a friend's post 402regarding a photo posted by a network contact and, as such, performs asearch indication in association with the post 402. Such a searchindication may be, for instance, hold or position a finger over thepost, hover a cursor over the post, select the post using a cursor, etc.Upon recognizing such a search indication associated with the post 402,capturing contextual information is automatically initiated. Contextualinformation may include, for instance, image data (e.g., tags designatedor assigned to the image), the user posting the image, date of imageposting, time of image posting, date and/or time of image capturing,recognition of scene in image, recognition of individuals in image(e.g., using facial recognition in association with network contacts),etc. In accordance with a search of the contextual information, or aquery automatically generated in association therewith, various searchresults 404 can be received and presented at the user device. Searchresults 404 may include, for instance, information about YosemiteNational Park (as identified, for instance, via photo tags), vacationplanning information, advertisements/deals associated with YosemiteNational Park, vacation planning, information about other individualsidentified in the photo, or the like. Further, in some embodiments,search results may include additional images 406. Such images may besimilar in nature or subject matter as image 402 or contain some or allof the individuals that are included in image 402.

FIG. 5 illustrates another example in which a queryless search isperformed using a user device in association with a social networkingapplication (e.g., a web-enabled application or non-web enabledapplication). Assume a user is using a social networking applicationrunning on a user device, such as a laptop, tablet, or other mobiledevice. The user device may present social networking content 500 on adisplay of the user device. The presented content may include text, animage, audio content, video content, and/or the like. The querylesssearch techniques described herein are applicable to any applicationthat displays or otherwise presents content to a user.

Now assume that a user is not interested in any particular post, but isgenerally interested in information that may be useful or interesting tothe user. Accordingly, the user performs a search indication generallyassociated with the social networking content 502. Such a searchindication may be, for instance, hold or position finger over an edge ofthe application or page, select an icon or button, etc. Upon recognizingsuch a search indication associated with the content 502, capturingcontextual information is automatically initiated. Contextualinformation may include, for instance, content of posts, image dataassociated with posted photos (e.g., metadata, users tagged, etc.),and/or the like. Assume that extracted contextual information indicatesthat a number of friends were posting about a particular company's stock(i.e., company A) and a location of photos posted by a friend (i.e.,Hawaii) as well as user information such as that the user is interestedin travel and investing. In accordance with a search using suchcontextual information, or a query automatically generated inassociation therewith, various search results 504 and 506 can bereceived and presented at the user device. For example, search result504 includes a stock quote for company A, and search result 506 includescontent related to Hawaii, such as tourist information, stateinformation, various beaches, etc.

Turning now to FIG. 6, FIG. 6 illustrates an example in which aqueryless search is performed using a user device in association with avideo hosting application (e.g., a web-enabled application or non-webenabled application). Assume a user is using a video hosting application600 running on a user device, such as a laptop, tablet, or other mobiledevice. The user device may present video content 602 on a display ofthe user device. The presented content may include or be associated withtext, an image, a video, a video clip, etc. For example, the user mayview a video or video clip presented by a web browser on the display asshown in FIG. 6. The queryless search techniques described herein areapplicable to any application that displays or otherwise presentscontent to a user.

Now assume that a user is interested in the particular video 602 beingviewed by the user and, as such, performs a search indication inassociation with the video 602. Such a search indication may be to, forinstance, hold or position a finger over the video, hover cursor overthe video, select the video using a cursor, etc. Upon recognizing such asearch indication associated with the video 302, capturing contextualinformation is automatically initiated. Contextual information mayinclude, for instance, a name of the video (e.g., identified via thecontent displayed or metadata), a time/date of the video post or videocapture, a poster's name or identifier, etc. In accordance with a searchusing the contextual information, or a query automatically generated inassociation therewith, various search results can be received andpresented at the user device. By way of example, search results 604pertain to information about the video (e.g., the name of the video, thename of an actor, actress, singer, etc.) and other similar or relatedvideos (e.g., videos of similar songs). Search results 606 pertain toposts from the user's network contacts that commented on the video 602.Search results 608 pertain to videos selected or viewed by the user'snetwork contacts. Any type of search results can be presented and thoseprovided in FIG. 6 are merely exemplary in nature. For example, searchresults may include an audio associated with the video, an option topurchase the video or audio, other related videos or audios (e.g.,related by poster, date, actor, producer, rating, subject matter, etc.),or the like.

FIGS. 7-9 illustrate an example of a queryless search performed inassociation with a television program. Initially, FIG. 7 illustrates anexample in which a television program 702 is presented via a displayscreen 704. Such a presentation might be facilitated, for instance, viaa set top box associated with the display screen 704. Assume that a useris interested in a particular individual receiving an award during anaward ceremony. As such, as illustrated in FIG. 8, the user 802 providesa search indication by providing a motion, such as, substantiallyenclosing a particular individual 804 receiving an award. Uponrecognizing such a search indication associated with the individual 804receiving the award, capturing contextual information is automaticallyinitiated. Contextual information may include, for instance, thetelevision program, the television station, the time of the searchindication, the time of the presentation of the television program, thechannel presenting the television program, the location of the user, theuser identifier, the user's profile, etc. In accordance with a searchusing the contextual information, or a query automatically generated inassociation therewith, various search results 904 can be received andpresented at the user device, as illustrated in FIG. 9. Search results904 may include, for instance, the name of the award recipient, a rolefor which the award was won, other's comments (e.g., critics), networkcontacts' comments, information about the awards program, informationabout other award winners, information about previous award winners,filmology information, images associated with the awards program or theaward recipient, videos associated with the awards program or the awardrecipient, movies and/or television programs the user has access to(e.g., via NETFLIX®, AMAZON®, etc.), an option to get connected tosocial network feed(s), chat(s), or group(s), or the like. Asillustrated in FIG. 9, in some embodiments, the search results canconsume at least a majority of the display screen. For instance, thetelevision program 902 has been reduced in size such that the searchresults can be prominently displayed. In other embodiments, the searchresults can be displayed on a smaller portion of the display screen(e.g., a side panel) or can be integrated with the television programcontent.

Exemplary methods for performing a queryless search are described withreference to FIGS. 10-12. These exemplary methods can be described inthe general context of computer executable instructions. Generally,computer executable instructions can include routines, programs,objects, components, data structures, procedures, modules, functions,and the like that perform particular functions or implement particularabstract data types. The methods can also be practiced in a distributedcomputing environment where functions are performed by remote processingdevices that are linked through a communication network or acommunication cloud. In a distributed computing environment, computerexecutable instructions may be located both in local and remote computerstorage media, including memory storage devices.

The exemplary methods are illustrated as a collection of blocks in alogical flow graph representing a sequence of operations that can beimplemented in hardware, software, firmware, or a combination thereof.The order in which the methods are described is not intended to beconstrued as a limitation, and any number of the described method blockscan be combined in any order to implement the methods, or alternatemethods. Additionally, individual operations may be omitted from themethods without departing from the spirit and scope of the subjectmatter described herein. In the context of software, the blocksrepresent computer instructions that, when executed by one or moreprocessors, perform the recited operations.

FIG. 10 illustrates an exemplary method 1000 of initiating a querylesssearch based on a search indication. The method 1000 is described in thecontext of the example user device for ease of illustration, but is notlimited to being performed using such a user device.

At block 1002, content is displayed on a display. By way of example andnot limitation, content may be displayed on a display, such as displayof the user device 202. The content may be displayed within anapplication, such as a web browser, email client, or any otherapplication on the user device. Additionally or alternatively, contentsuch as a video or an audio may be played by a media player of the userdevice. The content may include text, an image, a picture, audiocontent, video content, and/or the like.

At block 1004, a search indication is detected. The search indicationmay be a general search indication or a search indication associatedwith a particular item or content displayed. Such a search indicationmay include a touch input on a touch screen or touch pad, a body motiondetected by an image capture device, a motion of a cursor controlled bya user input device, or the like.

In response to detecting the search indication, at 1006, contextualinformation is captured. Such contextual information can be capturedusing various components of the user device or other computing devices.By way of example, captured contextual information may include GPS dataindicating the location of the user device; a user profile residing onthe user device or remote from the user device; a webpage or applicationbeing visited or used; previous user actions on the user device; a timeat which the search indication is received; a time, frame, or locationof media content presented when the search indication was detected; anindication of displayed media content (e.g., indication orrepresentation of an image or video); metadata associated with displayedcontent; author of a social networking post; individuals in an image,content of social networking posts, or the like.

At block 1008, the captured contextual information, or a portionthereof, is provided to a search service for performing a search usingthe contextual information. In this regard, a search using thecontextual information may be automatically caused to be performed inresponse to completion of the search indication. The contextualinformation may be sent to a remote search service, which is separatefrom the user device of the user. Additionally or alternatively, thesearch may be performed locally in the user device.

Thereafter, at block 1010, one or more search results are received fromthe search service. The one or more search results may include contentrelated to the contextual information captured in association with thesearch indication. The one or more search results are displayed on thedisplay of the user device. This is illustrated at block 1012. Thesearch results identified as relevant to the search query areautomatically displayed within the display screen. As such, a searchservice may return one or more search results relevant to the contextualinformation to the user device which may then present the one or moresearch results to the user. As can be appreciated, search results aredisplayed to a user independent of a user opening a web browser andnavigating to a search service. That is, a search result listing can bedisplayed upon detection of a search indication even though a webbrowser may not be open or active at the time the search indication isdetected. In some cases, placement of the search results may bedetermined. In embodiments, the user may interact with the searchresults to view details associated therewith and/or refine the searchresults. For instance, the user may select a search result to view moredata pertaining to the search result.

The search results may be displayed until a lapse of a time periodand/or an occurrence of an event. By way of example and not limitation,search results may remain active or displayed on the display screenuntil a predetermined time period has lapsed without use of the searchresults (e.g., user viewing, selecting, etc.). By way of furtherexample, a hide or collapse indication can be received, whicheffectively results in display of the original content withoutinterrupting the user's workflow.

Turning now to FIG. 11, FIG. 11 illustrates an exemplary method 1100 ofperforming a queryless search in accordance with a search indication.The method 1100 is described in the context of the search service forease of illustration, but is not limited to being performed thereon.

Initially, at block 1102, contextual information is received from a userdevice. Such contextual information may include, for example, GPS dataindicating the location of the user device; a user profile residing onthe user device or remote from the user device; previous user actions onthe user device; a time at which the search indication is received; atime, frame, or location of media content presented when the searchindication was detected; an indication of displayed media content (e.g.,indication or representation of an image or video); metadata associatedwith displayed content; an author of a social networking post; anindividual(s) in an image, content of a social networking post(s), orthe like.

At block 1104, a search query is formulated based upon the receivedcontextual information. Subsequently, at block 1106, a search isperformed or initiated utilizing the generated search query to identifyone or more search results. In embodiments, an Internet search may beperformed using the formulated search query. Any number of searches forsearch results can be performed, for example, a search for web results,a search for images, a search for media, a search for advertisements,etc. The one or more search results are provided to the user device, asindicated at block 1108.

With reference to FIG. 12, FIG. 12 illustrates an exemplary method 1200of initiating a queryless search based on a search indication receivedwhen media content is being presented. The method 1200 is described inthe context of the example user device for ease of illustration, but isnot limited to being performed using such a user device.

At block 1202, media content is displayed on a display. By way ofexample and not limitation, media content may be displayed on a display,such as display of the user device 202. The media content may be playedby a media player of the user device or a television associated with aset top box or other computing device. The media content may includevideo and/or audio media content.

At block 1204, a search indication is detected. The search indicationmay be a general search indication or a search indication associatedwith a particular item or content displayed. Such a search indicationmay include, for example, a touch input on a touch screen, touch pad, orremote control; a body motion detected by an image capture device;and/or a motion of a user input device.

Upon detecting the search indication, at 1206, contextual informationrelated to the presented media content is automatically captured. Suchcontextual information can be captured using or by accessing variouscomponents of the user device or other computing devices. By way ofexample, captured contextual information may include the location of theuser; a time, frame, or position of the media content at or near thetime of the search indication; an indication of displayed media content(e.g., indication or representation of an image or video); metadataassociated with displayed content, or the like. Contextual informationmay also include personal data associated with the user or data obtainedrelation to another user device.

At block 1208, the captured contextual information related to thepresented media content, or a portion thereof, is provided to a searchservice for performing a search using the contextual information. Inthis regard, a search using the contextual information may beautomatically caused to be performed in response to completion of thesearch indication. The contextual information may be sent to a remotesearch service, which is separate the user device of the user. Forexample, the contextual information may be sent to a traditionalInternet search engine to obtain more information about the mediacontent presented, such as the actor, a media summary or overview, anindication of other related media, etc. Additionally or alternatively,the search may be performed locally in the user device, for example, toobtain an indication of other media that the user has local access to(e.g., related media).

Thereafter, at block 1210, one or more search results are received fromthe search service. The one or more search results may include contentrelated to the contextual information captured in association with thesearch indication. The one or more search results are displayed on thedisplay of the user device. This is illustrated at block 1212. Thesearch results identified as relevant to the search query areautomatically displayed within the display screen. As such, a searchservice may return one or more search results relevant to the contextualinformation to the user device which may then present the one or moresearch results to the user. As can be appreciated, search results aredisplayed to a user independent of a user opening a web browser andnavigating to a search service. That is, a search result listing can bedisplayed upon detection of a search indication even though a webbrowser may not be open or active at the time the search indication isdetected. In some cases, placement of the search results may bedetermined.

Any of the acts of any of the methods described herein may beimplemented at least partially by a processor or other electronic devicebased on instructions stored on one or more computer-readable media.

Although the invention has been described in language specific tostructural features and/or methodological acts, it is to be understoodthat the invention is not necessarily limited to the specific featuresor acts described. Rather, the specific features and acts are disclosedas exemplary forms of implementing the invention.

The invention claimed is:
 1. A computer-implemented method offacilitating a queryless search, the method comprising: under control ofone or more processors configured with computer-executable instructions:recognizing a search indication provided by a user to indicate a desireto view one or more search results related to content being presented,wherein the search indication is performed without accessing a web pageassociated with a search engine; in accordance with the searchindication, automatically capturing contextual information related tothe content being presented; automatically providing the contextualinformation to initiate a search for information corresponding with thecontextual information; and receiving one or more search resultscorresponding with the contextual information.
 2. The method of claim 1further comprising causing the one or more search results correspondingwith the contextual information to be displayed.
 3. The method of claim2, wherein the one or more search results are overlaid on the contentbeing presented.
 4. The method of claim 1, wherein the search indicationcomprises a touch input on a touch screen, touch pad, or remote control;a body motion detected by an image capture device; a motion of a userinput device; a motion of a cursor controlled by the user input device;a selection controlled by the user input device; or a combinationthereof.
 5. The method of claim 1, wherein the search indication is ageneral search indication that does not include a specific item ofinterest within the content being presented.
 6. The method of claim 1,wherein the contextual information comprises information related to thepresented content, the user, a user device being used by the user, abehavior of a third-party deemed similar to the user, or a combinationthereof.
 7. The method of claim 1, wherein the one or more searchresults comprise one or more of text, images, video content, or audiocontent.
 8. One or more computer-readable media storing instructionsthat, when executed by a processor, configure the processor to performacts comprising: detecting a search indication comprising an actionperformed by a user while media content is being presented; in responseto detecting the search indication, automatically obtaining contextualinformation related to the media content at or near detection of thesearch indication; initiating a search to be performed using thecontextual information by automatically communicating the contextualinformation at least to a remote search service that formulates a querybased on the contextual information; and receiving one or more searchresults related to the obtained contextual information.
 9. The media ofclaim 8, wherein the acts further comprise presenting the one or moresearch results of the search by overlaying the one or more searchresults on a display screen displaying the media content.
 10. The mediaof claim 8, wherein the action is performed via a motion of the user.11. The media of claim 8, wherein the action is performed via a remotecontrol operated by the user.
 12. The method of claim 8, wherein thecontextual information comprises one or more of: text associated withthe media content; an identifier of the media content; a time associatedwith the search indication; metadata associated with the media content;a time associated with presentation of the media content; or a channelpresenting the media content.
 13. The media of claim 8, wherein thecontextual information comprises at least user activity associated witha remote user device.
 14. The media of claim 8, wherein the region ofcontent associated with the selector is selected at the time the searchgesture is performed.
 15. The media of claim 8, wherein the one or moresearch results comprise text information describing the media content, arepresentation of a similar or related media object, a social networkpost associated with the media content, or a combination thereof.
 16. Auser device comprising: a processor; memory communicatively coupled tothe processor; a display screen configured to display content; a searchdetector component to detect a search indication that provides anindication to perform a search using contextual information, wherein thesearch indication does not specify any text for use in performing thesearch; a context capturer component to automatically capture contextualinformation in accordance with detecting the search indication, whereinat least a portion of the contextual information corresponds with a timeat or near detection of the search indication; and a search presentingcomponent to cause presentation of one or more search results thatcorrespond with the contextual information, wherein the one or moresearch results are presented in association with the displayed content.17. The user device of claim 16, wherein a search is automaticallyinitiated by providing the contextual information to a search service.18. The user device of claim 16, wherein the one or more search resultsare identified based on only the contextual information automaticallycaptured in response to the search indication.
 19. The user device ofclaim 16, wherein a placement of the one or more search results isdetermined based on a predetermined location or an optimal location inaccordance with the displayed content.
 20. The user device of claim 16,wherein the search indication is provided by a user motion.